arXiv:1508.02108vl [cs.SY] 1 Aug 2015 


1 


Steady-State Performance of Incremental LMS 
Strategies for Parameter Estimation Over Fading 

Wireless Channels 

Azam Khalili, and Amir Rastegarnia 


Abstract —We study the effect of fading in the communication 
channels between nodes on the performance of the incremen¬ 
tal least mean square (ILMS) algorithm. We derive steady- 
state performance metrics, including the mean-square deviation 
(MSD), excess mean-square error (EMSE), and mean-square 
error (MSE). We obtain the sufficient conditions to ensure mean- 
square convergence, and verify our results through simulations. 
Simulation results show that our theoretical analysis closely 
matches the actual steady state performance. 

Index Terms —Adaptive networks, distributed estimation, in¬ 
cremental least mean square 


I. Introduction 

There are already several useful strategies for estimation 
over distributed networks. An example of a distributed method 
is the consensus strategy 0-0 in which each node performs 
a local estimation and fuses its estimate with those of its 
neighbors so that all nodes converge to the same estimate 
as the number of iterations increases. The main problem of 
mentioned methods is that the framework does not allow the 
network to undertake a continuous learning and optimization 
0 which motivated the development of adaptive networks. An 
adaptive network is a collection of agents (nodes) that collabo¬ 
rate with each other through in-network local processing rules 
in order to estimate and track parameters of interest 0 - Two 
major classes of adaptive networks are incremental strategy 
0-0 anc * diffusion strategy [?], (TTJ, JT2j . In the incremental 
mode, nodes passing updates to each other in a Hamiltonian 
cycle in the network, while in diffusion mode, each node can 
communicate with a subset of neighboring nodes. 

In this paper, we investigate the performance of the ILMS 
algorithm in a wireless adaptive network with communication 
links between neighboring nodes modeled as fading channels. 
In the original ILMS strategy proposed by JT3) , it is assumed 
that nodes communicate with each other via ideal links which 
is typically not true in practice. In 114| 118|. the effect of 
additive link noise in the communication channels between 
nodes have been investigated. In 0D’ the performance of 
general adaptive diffusion algorithms in the presence of im¬ 
perfect information exchanges, including quantization errors, 
and model non-stationarities has been considered. All these 
works however can not be directly applied to a wireless 
adaptive network, whose communication links are modeled 
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by fading channels [201. The references [ |211 , |22) propose 
diffusion LMS algorithms for wireless sensor networks with 
fading channels but under the assumption that channel state 
information is known so that channel equalization can be 
performed. 

In this paper, our objective is to investigate how the perfor¬ 
mance of the ILMS algorithm is affected by the channel fading 
statistics of the communication channels between nodes. We 
show that the ILMS algorithm over fading channels, like the 
traditional ILMS algorithm, is stable in mean if the step size 
is chosen in an appropriate range. This range now not only 
depends on the regression correlation, but also on the mean 
of the channel fading gains. Moreover, our analysis reveals 
that in general, fading communication channels lead to biased 
estimates in steady state. We derive closed-form expressions 
for the steady state performance metrics, including the mean- 
square deviation (MSD), excess mean-square error (EMSE) 
and mean-square error (MSE), of the ILMS algorithm in the 
presence of fading channels, and under a Gaussian model. 
We show explicitly how these metrics are affected by the 
fading channel statistics. We derive sufficient conditions for 
the convergence of the MSD, EMSE and MSE, and show that 
for a fixed step size, mean-square stability is lost if the channel 
gain variances become large. We present simulations to verify 
that our theoretical analysis closely matches the actual steady 
state performance. 

Notation-. We adopt boldface letters for random quantities. 
The symbol * denotes conjugation for scalars and Hermitian 
transpose for matrices. The notation diag{-} will be used in 
two ways: X = diag{x} is a diagonal matrix whose entries are 
those of the vector x, and x = diag{X} is a vector containing 
the main diagonal of A'. If X is a matrix, we use the notation 
||x|||, = x*Sx for the weighted square norm of x. If a is a 
vector, the notation ||at|| CT is used to represent ||x||di ag {cr}- 

II. ILMS Algorithm over fading channels 

Consider a network composed of N nodes. At time i , node 
k observes a scalar measurement dk{i) and alxM regression 
vector Uk which are related via a linear regression model 

d k (i) =u k ,iw°+ v k (i), ( 1 ) 

where Vk(i) is the observation (or measurement) noise, and the 
Mxl vector w° is a unknown vector. The goal of the network 
is to estimate w°, at every node k, using all observed data in 
the entire network. In the incremental LMS algorithm 
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each node k receives a local estimate from the previous node 
k — 1, updates it using its local data, and then sends it to the 
next node k + 1. The update equations for the ILMS algorithm, 
at iteration i is given by [6| 

f w 0 ,i <- w N}i -1 

\ w k ,i =w k -i,i +^kU* k i {dk{i) -u k ,iW k -i,i) 

where Wk,i is the local estimate of the node k at time i. It is 
shown in [6| that Wk,i —> w° as * —► oo for every node k. 

Now, we consider the case where nodes communicate 
over fading channels. By incorporating the impact of fading 
channels, the update equation for node k in ([2]) becomes 

Wk,i = hk(i)v>k-i,i+q k ,i 

+ ( d k {i ) - u k ,i(h k (i)w k -i,i + q k>i )) , (3) 


where h k (i) is the channel gain at time i for the communica¬ 
tion channel between node k — 1 and k, and q k i is the zero 
mean channel noise with covariance matrix Q k = E[q k t q k J. 
We assume phase coherent reception at every node k and 
model the channel gain as a non-negative random variable. We 
make the following assumptions regarding the fading channel 
statistics. 

Assumption 1: 

1) The channel gains h k (i ) for all nodes k = 1,N, and 
all observation times i > 1, are independent of each other. 
For each node k , the channel gains {h k (i) : i > 1} are 
identically distributed. 

2) The channel gains h k (i) for all nodes k = 

and all observation times i > 1, are independent of 
(di (j), Uj j ) for all l and j. 


Let m k = E[/ifc(z)] and s k = E h k (i) be the mean 
and second order moment of the channel gain for node k, 
respectively. We note that in practice, even if the nodes 
perform channel state estimation, it is not possible to measure 
the channel gains with absolute certainty, especially if the 
channels experience fast fading. In this case, without loss of 
generality, the channel gain for node k can still be modeled 
as a non-negative random variable with a non-trivial variance. 


and the weight error vector, which is the deviation between 
the local estimate w k ,i and its true value w°, given by 

W k ,i = w° - W k ,i■ 

By subtracting w° from both sides of 0 and using the 
definition of w kk we obtain 

Wk,i — ^k{t)W k —X^i 

+ (1 - h k (i))w° - p, k h k (i)u* k i u kti w k -i,i 
- p k u* ki v k {i) - p k (l - h k (i))u k jU k ^w° - q k} i 
+ p k u* kl u k ,iq k}i . (4) 

In the steady-state analysis, we are interested to quantify the 
performance using the following metrics at every node k: 


r] k = lim E [||ti) fe _i j|| 2 ] 

i—> 00 

(MSD) 

(5) 

Ck = lim E 

i—t 00 

61 

?r 

1 

i,k 

(EMSE) 

(6) 

£ k = lim E [|e fc (?.)| 2 ] = 
2—>-00 

Cfc + < fe (MSB) 

(7) 


To derive the above steady state performance metrics, we need 
to evaluate quantities of the form E [\\w k 4IIIJ where E fc 
is a positive semi-definite Hermitian matrix. To this end, we 
consider the weight vector update equation given by Let 

, i tTl k J k C' k _i i "L (1 Tl'l k )J k i (8) 

where J k = I - p k R u ,k, Co,* = Cjv,*-i, and C 0 ,i = I. Note 
that the matrix C kt i is such that 


E [w fcjl ] = C k>i w°. (9) 

By equating the weighted norm of both sides of Q. taking 
expectations and using Assumptions [2] and [I] and 0, we 
obtain the following recursive relationship: 


E [\\w kti 


IIJ 


IIumIIgiJ + 


MfcO- * 2 ,feE 


U k,. 


lit 


\T k +H k<i , 


( 10 ) 


where 


III. Performance Analysis 

In this section, we analyze the mean stability and steady 
state mean-square performance of the ILMS algorithm when 
communication channels between nodes are fading channels. 
Our analysis is based on the energy conservation approach of 
|6). We make the following assumptions regarding the data 
model in Q. These assumptions are commonly assumed in 
the literature UD- 

Assumption 2: 

(i) The regression vectors u k> i are independent over node 
indices k and observation times i. 

(ii) The measurement noises v k (i) are independent of each 
other and the regression vectors u k ,i- 

In our analysis, we will use the deviation between an 
observed measurement and its prediction based on the current 
local estimate, which is defined as 

6k(t') — d k (f) U k ^iW k —X } ii 


G k — E/c /t/cE ^Yj k u k ^u k ^i T u k jU k ,iYj k j (11) 

+ plE [|K,*lll fc u fc,* u M] 

K = s k G k (12) 

T k = (1 — 2m k + s k )G k (13) 

H k ,i = ( tn k — s k )(C k -x,iG k + G k C k -x,i)- (14) 


In order to compute all the moments that appear in the recur¬ 
sive equation © and to obtain closed-form expressions, we 
now make the following assumption regarding the regression 
vectors u k , for all nodes k = 1,..., N. 

Assumption 3: For each k = 1..... iV, the distribution of 
u k is a Gaussian distribution with 

R u ,k = U k A k U* k , (15) 

where A k is a diagonal matrix with diagonal elements being 
the eigenvalues of the correlation matrix R Utk , and U k is 
unitary matrix. 
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Making use of Assumption [3] we further define the follow¬ 
ing transformed quantities: 

w k ,i = U k w kt i, Sfc = U£Y k U k , (16) 

E'fc = UZJy k U k ,u k)i = u k ,iU k , (17) 

T k = U* k T k U kl H kti = U k H k i U k , (18) 

w° = U k w°, C k -i ti = U* k C k . 1}l U k , (19) 

Qk = U k Q k U kl D = w°w°* = w°w°*. (20) 

From the above definitions, equation © can now be 
rewritten in the following equivalent form 


E 


IKillL 


= E 


IWfe-l.iHs 

+ E 


9k,i\\e k 


2 2 w 

Mfc°T,fcE 

o II2 




+ \w u 


I Tk+Hk,i 


where in ( |2Tj ) we have 

G k = Yj k /TfcE ^Yj k U k ^U k ,i -{- U k j1l k ,iYj k j 

+ fx k Ej\\u k ^\\j, k U ki u ki i 

= Efc [j, k (Yj k A k T- A k Yl k ) 

+ Atfe(-^-fcTr[EfcAfc] + "/A k Y, k A k ) 

E& = s k G k 

T k = (1 - 2m fe + s fe )G fc 

H k ,i = {m k — s k )(C k -i : iG k + G k C k -i,i) 

= 2 (m k - s k )C k -i,iG k 

Further algebraic manipulations of ([2T| yields 


E 


= E 

ills' 

k. 

+ Fk a v,kF^[Ak'Fj k \ 



+ Tr [Q k G k ] + Tr [DT k ] + 1 X[DH kti ]. (25) 


s k p(F k ) < 1. 


Suppose that s k = s for all nodes k, and the step sizes fi k are 
fixed. Then, if s is sufficiently large, the left hand side of ( [28] ) 
diverges and we no longer have mean-square stability. This 
shows that deteriorating fading conditions have detrimental 
impact on the ILMS algorithm, and care should be taken to 
adjust the step sizes according to ( |30[ . 

Assuming that step sizes are chosen sufficiently small, and 
by letting i —>• oo, the recursive equation ( |28[ ) at steady-state 
gives 


E [||tfl fc ,o 


\lu] 


= E 


\w k - 1, 


+ gkcr k , (31) 


where 


( 21 ) 


9k = l4 a l, k x k + diag {Q k } T F k 
+ (1 - 2 m k + s k )diag{D} T F k 
+ 2{m k - s fe )diag{D} T Cfc — 1,00 F k , 
Moreover, C k _ ljOC in ([32]) is given by 


( 22 ) 

(23) 


(24) 


To derive (|5]i-(|7|, we only need to consider the case where T, k 
is a diagonal matrix. In this case, matrix T! k is also a diagonal 
matrix. We let 

CT fc = diag{E fc }, d' k = diag{E' fc }, \ k = diag{A fc }, (26) 
and 

F k = I — fi k X k + P k Y k , (27) 

with X k = 2Afc and Y k = Al + X k X k . The M x M matrix F k 
contains the statistics of data local to node k. We then have 

E [II^mIIU = E +9k,id k , (28) 

where g kt i and a' k are given respectively by 

9k,i = Fk^v^^k F diag{Qfc} F k 
+ (1 - 2 m k + s k )diag{D} T F k 
+ 2 (m k - s k )d\ag{D} T G k -\,iF k , 
a k = s k F k (j k . (29) 

We can use ( [28] ) to derive conditions that guarantee conver¬ 
gence in the mean-square sense for the ILMS algorithm with 
fading channels. Under assumptions [2] [T| and [3] the ILMS 
algorithm over fading channels converges in the mean-square 
sense if the step sizes fi k are chosen to be sufficiently small 
so that the 


(32) 


Gfc-1,00 = U k ( (J - Ad ) -1 x 

W(l-TOn)J„ J] rn e J e j\u k . ( 33 ) 

n —1 ^ t=n +1 ) 

We observe that © shows how E [HtUfc.oolllJ evolves 
through the network, which in its current form makes it diffi¬ 
cult to derive the desired metrics 0-0 directly. In fact, we 
have to find a recursive equation that reveals how E [||u>fc,i||| fc ] 
evolves in time. By iterating ( [28] ), and using w o,i+i = wN.i., 
we can obtain a set of N coupled equations. With suitable 
manipulation of these equations, along with proper selections 
of a k , it is possible to solve the resulting equalities to derive 
the desired metrics. Following the argument given in |6j, we 
can derive the required metrics in a similar way as 

Tfk = a k {I - n M ) -1 l ( 34 ) 

Cfc = a k{I - n fc ,i) -1 Afc ( 35 ) 

6 = Cfc + <k (36) 


where 


n k ,i = 


N 

II Sfe 

fc=1 


F k +i-iF k+ i ■ ■ ■ F n Fi ■ ■ ■ F k _i , (37) 


(30) 


a k = 9kJdk,2 + gk+l^dk,3 + • • ■ + g k -2^dk,N + gk-1, ( 38 ) 

where l = 1, ■ ■ • , N and all the subscripts are in mod N. 

IV. Simulation Results 

We illustrate the results via simulations. We assume a 
network composed of N = 20 nodes, where the nodes are 
connected via a ring topology as in the ILMS algorithm. The 
regressors u k i are generated as independent realizations of 
a Gaussian distribution with a covariance matrix R u , k whose 
eigenvalue spread is 5. The measurement data d k (i) at each 
node k is generated by using the data model ([T]) where the 
parameter w° is chosen to be [1 1 1 l] T /2, and the observation 
noise v k (i) is drawn from a Gaussian distribution with vari¬ 


ance erf; u as shown in Figure [T] The additive channel noises 
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Fig. 1. Node profile and channel noise information: crj k (up), cr^ k (middle) 
and Tr (R u *.) (down). 


are generated from Gaussian distributions with covariance 
matrix Qk = a for k = 1, ...,20. The values of a 1 k 
are shown in Figure [I] We generate the channel gains hk(i) 
using a Rayleigh distribution with m= v/2/2 for all values 
of k. To obtain the steady-state values of MSD, EMSE and 
MSE, we run the ILMS algorithm with 2000 iterations and 
average the last 200 samples. Finally, each steady-state value 
is obtained by averaging over 100 independent runs. 

In Figure [2] we show the steady-state performance metrics 
MSD, EMSE and MSE as functions of the node index k when 
the step size /i = 0.02. We can see that the simulated results 
closely match the theoretical results. 
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Fig. 2. Steady-state curves versus for each individual node k, [i = 0.02. 


V. Conclusion 

In this paper, we have investigated the steady state per¬ 
formance of the ILMS algorithm when the links between 
nodes are fading channels, and we do not have perfect channel 
state information. Our analysis reveals how the mean-square 
stability depends on the channel gain variances. We also 
derived steady state performance metrics, including the MSD, 
EMSE and MSE. We present simulation results to verify our 
theoretical analysis. 
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